Video Based Human Emotion Estimation

This paper addresses the problem of human emotion estimation. Human emotion understanding will play a very important role in future humancomputer interaction systems. Human emotion is a complicated temporal behavior. The technical difficulty in human emotion estimation lies in that the inherent emotional states are not measured directly. The only way to estimate the emotional states is from observable signals, such as facial expressions, voice, and gestures, which are closely related to the emotional states. In this paper, we show how to make use of facial expressions to infer inherent emotional states. Hidden Markov Models are used to model the dynamic facial events which characterize human emotions. Facial expression parameters, action units, are extracted from video sequences containing faces and used as observations for the underlying Hidden MarkovModels. This methodology has been used to detect fatigue, which can be treated as an emotional state with associated typical facial events. Promising results are reported.